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(57) Abstract: To avoid the need for all the nodes of a network to know the load status of all other nodes and links in the network, the 
^ nodes (101....303) are organized in node groups (10, 1 1, 12, 20, 30). Each group has a master node (104, 124, 1 13, 201, 301) which 

is incorporated in a logical higher order group. By recursively grouping the units of each preceding group, a logical hierarchical 
^ structure is formed in which the number of units at each level (A, B, C, D), decreases. Each node informs the other nodes of its 
^ load status, and each higher level unit also exchanges information with the other units. The recursive, hierarchical structure greatly 

reduces the amount of load status information exchanged across the network. 
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RECURSIVE TRAFFIC DISTRIBUTION IP/DATA NETWORK MODEL 
Technical field ^: v ^ 

This invention relates to a method and arrangement for transferring network 
node status information between nodes. 
5 Background Art t , - f , . .... , ; 

Least cost routing examines the netwprk topography to determine the shortest 
path between the source of a message and its destination. This method does not 
take account of the load status of the individual links. of the chosen path. so one or .. 
more of the links may become overloaded,, preventing or; disrupting the delivery of 
1 0 the message to the destination. 

An alternative proposal is to take into account the load status of the links 
when determining the chosen path. Thus the system may determine all the shortest 
paths and make the path selection on the basis of the path with the links carrying 
the least traffic. To implement this, it is necessary for all the nodes to exchange load 
15 status information. As the number of nodes and links in a network grow, the 

implementation of this technique requires the exchange of a large amount of load 
status traffic, as each node broadcasts the load status of its associated links to the 
other nodes. 

In our co-pending application number 44470/99 (1 27045 SY), we disclose 
20 a network in which the message is spread over all the practical paths between the 

source and destination. An advantageous embodiment of that invention involves the 
elimination of paths having heavily loaded links. Again, the implementation of this 
technique requires the exchange of load status, information between nodes, 
generating a large volume of traffic. 
25 Disclosure of the Invention , _ " 

This specification discloses a network arrangement for a plurality of nodes 
each node being connected to one or more other nodes by corresponding node 
links, 

the network being arranged into a recursive hierarchy of units having 
30 two or more levels, 

the nodes being the units of the first level of the hierarchy, 
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the uni»i)£.higher levels of the hierarchy being formed by groupings 
of the units of the previous level, 

■ . * wherein the units of a level exchange a corresponding load status 
information. ' 
5 This invention can be used in conjunction with the maximal flow techniques 

described in : 44470/99 to determine suitable paths with available' capacity. 
Brief Description of the Drawings - M ! r " . • 

- Figure ! is a schematic re a network in which the nodes are 

arranged in a 'rfecursive hierarchy, in accordance with an embodiment of the 
10 invention. 

' Figure 2 represents a load status monitor. : ■ •• '- s ; 
* Figure 3 illustrates' the 1 message structure for exchanging information at 
different levels. n •' : ; 

Best Mode of Carrying out the Invention " * ' • #- 

15 Figure 1 shows a network of nodes interconnected by links. According to the 

embodiment shown in Figure 1, the nodes are linked in a logical hierarchy. 

The nodes exemplified by the circles, some of which are numbered 1 0V 

202 ... 303 are shown interconnected by node links, represented by the lines 

drawn between the nodes." 
20 The nodes are formed into groups 10, 1 T, 12, 20, 30. The groups are 

interconnected by group links, forexample 1001 between node : 105 of group 10 

and node 121 of group 12. 

The group links are shown in the following Table 1 . 



TABLE 1 : GROUP LINKS 



Node/Group 


Node/Group 


Link 


105 


10 


121 


12 


TOOT' 


112 


11 


122 


12 


1002 


103 


10 


111 


1 1 


1003 


123 


12 


202 


20 


1201 


101 


10 


302 


30 


1301 


102 


10 


303 


30 


1302 
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Preferably the group links havfe a larger traffic capacity than node links. 
Group links may equate to regional trunks within a particular carrie^s^etwork, or to 
links between different carriers, different countries or different global regions, for 
example, .r 
5 As shown in Figure 1 , the levels of the hierarchy start with the nodes. The 

units of the second level are the groups of nodes. The units 131 ; L32, L33, of the 
third level are one or more groups, and the units of the fourth level, L41, L42 are 
formed by aggregating units of the third level. Thus L41 is the aggregation of L31 ~ 
and L33, while L42 encompasses L32. 
1 0 Preferably the nodes are grouped on the basis of communication path 

topography so that there are relatively few links in the shortest path between any two 
nodes in a group. 

The groups themselves, 10, 11, 1'2, 20, 30 are linked by designated links 
1001, 1002, 1003 1201, 1301, 1302 connecting designated nodes in the 

15 corresponding groups. ' . ^ 

' Within each group, a master node is assigned, 104, 1 13, 124, 201, 301. 
Because each node in a group knows the load status of all the links in that group, 
the master node has the information to enable it to compile available capacity 
reports adapted to meet the requirements of the higher levels. 

20 The physical node interconnections are illustrated at A in Figure 1 and B,C 

and D illustrate the conceptual logical links for the second, third and fourth levels of 
hierarchy. 

At level A, the nodes of each group communicate load status information to 
each of the other nodes of the corresponding group. 
25 Thus the master nodes 104, 1 13, 124, of the network 1 shown at B are 

aggregated under the supervision of a single master node 113 which is assigned to 
the next higher level. Because, in the example shown, networks 2 and 3 have only 
one group, the same master 201, 301 is used all levels. 

At level C, the master nodes 1 13, 201 > 301 exchange information as to the 
30 overall load status of their associated networks 1, 2, and 3. 
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The podes qt level ,G pre then ,10 3, ,201 , 30 1 -By then aggregating 1 1 3 and 
301 / pne : of these tv/0-Gan be designated to manage the level D information , 
exchange for both netvyork 1 and network 2. Thus at level D, 201 and 301 can 
exchange information on the available capacity for the regions covered by L42 and 
5 L4.1: . .... . \ , 1? . . > . • - 

At level C/ infoririqtionon the capacity, of networks 1, 2 and 3 is exchanged 
between the netwprks,. At-level B, information on the capacity of. groups 10, 11, 12, 
20, ,30 is exchanged between/the grqups. \ y . 

At level A, the nodes interchange information . on traffic capacity at the node 
10 link level, within the, groups.; , . ' -. 

Figure 2 ; shows an arrangement for monitoring the available capacity of the 
links connected to a node. 

For each link connected to a node.there is a buffer 51, 52, 53, e.g. in the ; 
form of FIFO, . .. .. . 

1 5 The traffic level monitor 50 checks the level of the contents, of the buffers to 

measure the available capacity on the basis of the speed of the link associated with 
the buffer. The result of the monitoring is then reported to the other nodes in the T . 
same group. 

In a simplified measuring system, the monitor may report whether.or not a 
20 link has spare capacity, e.g. by checking whether a buffer's content is, above or 
• below a predetermined threshold. • 

The load status information exchange is carried out on the following basis. 
- The nodes within a group, each notify the : other, nodes within that group of the 
load status of the links connected to the notifying node. 
25 At the group level, each group notifies. the other groups of the load status of 

the links connected to the notifying group and a summary of the load status of 
internal paths within the group available for interconnecting the group links 
connected to the notifying group: 

For example, Group 12 is connected to Group 20 via link 1201, to Group 10 
30 via link 1001, and to Group 11 via link 1002. 
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. Preferably, the designated as a master node manages the interchange of 
information between the groups. 

Table 2 shows the master nodes for each group. , . ; 
TABLE 2 

5 



GROUP 


MASTER NODE 


10 


. 104 


11 


113 


12 


124 


20 


201 


30 


301 



As can be seen in Figure 1, the master nodes take part in the higher level 
exchanges but their number is progressiyely reduced by the recursive grouping. 

Thus, in the embodiment shown in Figure 1 , while there are 5 master nodes 
10 shown at level B, there are only 3 at level C and 2 at level D. 

Preferably, the grouping is carried out on the basis of proximity in the sense 
of the number of links in the path. Of course this is not a strict rule at the node level 
because the nodes at either end of a group link are joined by a single link, while 
there may be more than 2 links between nodes within a group. Other factors which 
1 5 influence grouping are geographical proximity and network ownership, as well as 
the traffic flows. ... 

For example, the nodes of network 2 may be geographically close to node of 
network 1, but network 1 may be owned by a different carrier from network 2. 

At level P, the nodes 201 and 301 exchange information on the available 
20 capacity between network 3 and network 2 and the transit capacity of the respective 
networks. This information would, for example, be based on the load status of links 
1201, 1301, 1302, and the capacity across network 1 between link 1201 and the 
links 1301, 1302. The information need only identify the maximum available 
capacity at the time, which varies in accordance with the load on the various 
25 network elements. 
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For the sake of efarity the information will be given the following names: 

Level D = Regional; 

Level C = Network; 

Level B = Group; 
5 Level A = Node. 

Regional information may be, for example, the maximum available capacity 
between the "electrically" remotest groups. The term "electrically" refers to the 
number of links and may include cable, optical and radio links. 

Network information may be, for example, the capacity between the various 
10 networks, including the trans-network capacity between the network links 1201, 
1301, 1302. 

Group information could be typified by the capacity between groups, 
including the trans-group capacity between the group links. 

Node information is the information broadcast by a node to the other nodes 
15 within its group as the load status of the node and its associated links. 

Group information can be deduced from node information. Each node in a 
group knows the load status of all the nodes in that group. Thus the master none 
124 in group 12 knows the status of group links 1001 from node 121, group link** 
1002 from node 122, and group/network link T201 from node 123, as well as the 
20 status of all the infernal nodes arid links within group 12. Node 124 can therefore 
calculate the available capacity across the group 12 between bny pair of the links 
1201, 1001, 1002. Preferably the master node 124 would use the -'all practical 
paths 77 algorithm of our Australian Patent application 44470/99 (Docket No. 
127045 SY) to calculate the trans-group capacity. This group information is 
25 interchanged 1 between the group master nodes 201, 124, 113, 104, 301 at level B. 

The units of the level B group domain are again grouped together, in this 
embodiment, into 3 network groups. The network groups include two one member 
groups 201 and 301, and one three member group 124, 1 13, 104. The network 
master of each one member group is the member of the group, while 1 13 is 
30 designated as the master of the three member network group. 
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The three network masters from leveF B interchange network information at 
the level C network domain. The information relates to the 'ribf^^lclih^'tonheding 
the respective networks, and the trans-network information relating to 'the capacity 
between the pairs of network links. 
5 At the regional domain, level D, the network masters 201 , 1 1 3, 307 have 

been formed into two groups, resulting in two regional masters 201, 301 , which 
exchange information on the available capacity between the two regions. 

The regional master nodes 201, 301, convey the regional link capacity 
information to the other regional nodes. In the present embodiment 301 conveys the 
10 information to 113. 201 is the only regional node in the other regional grouping. 

The regional nodes 201, 1 1 3 and 301 are all network master nodes and they 
convey the inter-regional and inter-network capacity information to the network level 
nodes. In our embodiments, 1 13 conveys this information to the nodes 104, 124. 

Each of the network level nodes 201, 124, 113, 104, 301 is a group master 
1 5 and relays the higher level information to each of the nodes in its group. 

The grouping of the units at each level means that the information exchanged 
at each level becomes more generalised. 

This means that a node has detailed capacity information about the other 
nodes in its group. Capacity information about other groups in its network, capacity 
20 information about the other networks in its region, and information about the inter- 
regional capacity. 

In a preferred embodiment the group master handles the interchange of 
node link capacity information. Each node, instead of broadcasting its load status to 
all the other nodes in the group, sends the information only to the group master, 
25 which collates the information from each node and relays the information to the 

other nodes. The message from the group master preferably incorporates the higher 
level load status information, so that each node has an overall picture of the entire 
system. 

Thus the group master may broadcast a message including the information 
30 shown in Figure 3. The first segment RL includes the load status at the regional link 
level D. A second portion of the payload includes a number of segments of 
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information onjhe internetwork lopd; status, NL A. third portion includes- segments 
GL.on the. i^ertgraupjloqd status, and the fourth. portion includes segments NL on 
the load status of the podes within the group. Alternatively, this information can be 
flooded to other part of the network using other means, such as a broadcast or 
5 multicast mechanism. ......... 
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The claims defining the invention are as follows: •.> - 

1 . A network arrangement for a plurality of nodes .each node being connected 
to one or more other, nodes by corresponding node links, - . , , • . 

the network being arranged into a recursive hierarchy of units having 
5 two or more levels, 

the nodes being the units of the first level of the hierarchy, 
the units of higher levels of the hierarchy being formed by groupings 
of the units of the previous level, 

wherein the units of a level exchange a corresponding load status 

10 information. 

2. An arrangement as claimed in claim 1 wherein, within, each group of units, a 
master entity is designated, the master entity conveying inter-unit load status 
information relating to the units of that level to the next higher level. 

3. An arrangement as claimed claim 1 or claim 2 wherein, in the first level, a 

1 5 selected node in each group is designated as the master node for the corresponding 
group, 

the master node managing the transfer of node load status 
information within its corresponding group. 

4. An arrangement as claimed in claim 1 or claim 2, or claim 3 wherein the 
20 load status information includes information on the available traffic capacity 

between the ports of each unit. 

5. An arrangement as claimed in any one of claims 1 to 4 wherein each node 
includes node load status monitoring means to monitor the load status of the links 
connected to the node. 

25 6. An arrangement as claimed in any one of claims 1 to 5 wherein at least one 
node of each second level group is connected to a node of at least one other second 
level group via a corresponding group link whereby group load status information 
can be interchange. 

7. An arrangement as claimed in claim 6 wherein the units of the third level are 
30 formed by mutually interconnected second level units. 
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8. A network arrangement for interchanging load status information 
substantially ^ dW fiereifi described with* reference to the accompanying drawings. 

9. A network arrangement -ds clairned in any one of claims 1 to 8 implementing 
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